{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(basic-demo)=\n",
    "# Feature store example (stocks)\n",
    "\n",
    "This notebook demonstrates how to generate features and feature-sets, build complex transformations and ingest to offline\n",
    "and real-time data stores, fetch feature vectors for training, save feature vectors for re-use in real-time pipelines,\n",
    "and access features and their statistics in real-time.\n",
    "\n",
    "```{admonition} Note\n",
    "By default, this demo works with the online feature store, which is currently not part of the Open Source MLRun default deployment.\n",
    "```\n",
    "\n",
    "**In this section**\n",
    "- [Get started](#get-started)\n",
    "- [Create sample data for demo](#create-sample-data-for-demo)\n",
    "- [Define, infer and ingest feature sets](#define-infer-and-ingest-feature-sets)\n",
    "- [Get an offline feature vector for training](#get-an-offline-feature-vector-for-training)\n",
    "- [Initialize an online feature service and use it for real-time inference](#initialize-an-online-feature-service-and-use-it-for-real-time-inference)\n",
    "\n",
    "**See also**\n",
    "- [**Fraud prevention demo**](https://github.com/mlrun/demo-fraud): Use the feature store to process raw transactions and events in real-time and respond and block transactions before they occur.\n",
    "\n",
    "## Get started\n",
    "\n",
    "Install the latest MLRun package and restart the notebook.\n",
    "\n",
    "Setting up the environment and project:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "> 2024-09-11 12:37:21,186 [info] Server and client versions are not the same but compatible: {'parsed_server_version': Version(major=1, minor=7, patch=0, prerelease='rc40', build=None), 'parsed_client_version': Version(major=1, minor=6, patch=3, prerelease=None, build=None)}\n",
      "> 2024-09-11 12:37:21,273 [info] Project loaded successfully: {'project_name': 'stocks'}\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<mlrun.projects.project.MlrunProject at 0x7f5736f43e80>"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import mlrun\n",
    "\n",
    "mlrun.get_or_create_project(\"stocks\", \"./\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create sample data for demo"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "tags": [
     "hide-cell"
    ]
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "\n",
    "quotes = pd.DataFrame(\n",
    "    {\n",
    "        \"time\": [\n",
    "            pd.Timestamp(\"2016-05-25 13:30:00.023\"),\n",
    "            pd.Timestamp(\"2016-05-25 13:30:00.023\"),\n",
    "            pd.Timestamp(\"2016-05-25 13:30:00.030\"),\n",
    "            pd.Timestamp(\"2016-05-25 13:30:00.041\"),\n",
    "            pd.Timestamp(\"2016-05-25 13:30:00.048\"),\n",
    "            pd.Timestamp(\"2016-05-25 13:30:00.049\"),\n",
    "            pd.Timestamp(\"2016-05-25 13:30:00.072\"),\n",
    "            pd.Timestamp(\"2016-05-25 13:30:00.075\"),\n",
    "        ],\n",
    "        \"ticker\": [\"GOOG\", \"MSFT\", \"MSFT\", \"MSFT\", \"GOOG\", \"AAPL\", \"GOOG\", \"MSFT\"],\n",
    "        \"bid\": [720.50, 51.95, 51.97, 51.99, 720.50, 97.99, 720.50, 52.01],\n",
    "        \"ask\": [720.93, 51.96, 51.98, 52.00, 720.93, 98.01, 720.88, 52.03],\n",
    "    }\n",
    ")\n",
    "\n",
    "trades = pd.DataFrame(\n",
    "    {\n",
    "        \"time\": [\n",
    "            pd.Timestamp(\"2016-05-25 13:30:00.023\"),\n",
    "            pd.Timestamp(\"2016-05-25 13:30:00.038\"),\n",
    "            pd.Timestamp(\"2016-05-25 13:30:00.048\"),\n",
    "            pd.Timestamp(\"2016-05-25 13:30:00.048\"),\n",
    "            pd.Timestamp(\"2016-05-25 13:30:00.048\"),\n",
    "        ],\n",
    "        \"ticker\": [\"MSFT\", \"MSFT\", \"GOOG\", \"GOOG\", \"AAPL\"],\n",
    "        \"price\": [51.95, 51.95, 720.77, 720.92, 98.0],\n",
    "        \"quantity\": [75, 155, 100, 100, 100],\n",
    "    }\n",
    ")\n",
    "\n",
    "\n",
    "stocks = pd.DataFrame(\n",
    "    {\n",
    "        \"ticker\": [\"MSFT\", \"GOOG\", \"AAPL\"],\n",
    "        \"name\": [\"Microsoft Corporation\", \"Alphabet Inc\", \"Apple Inc\"],\n",
    "        \"exchange\": [\"NASDAQ\", \"NASDAQ\", \"NASDAQ\"],\n",
    "    }\n",
    ")\n",
    "\n",
    "import datetime\n",
    "\n",
    "\n",
    "def move_date(df, col):\n",
    "    max_date = df[col].max()\n",
    "    now_date = datetime.datetime.now()\n",
    "    delta = now_date - max_date\n",
    "    df[col] = df[col] + delta\n",
    "    return df\n",
    "\n",
    "\n",
    "quotes = move_date(quotes, \"time\")\n",
    "trades = move_date(trades, \"time\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### View the demo data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "tags": [
     "hide-output"
    ]
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>time</th>\n",
       "      <th>ticker</th>\n",
       "      <th>bid</th>\n",
       "      <th>ask</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2024-09-11 12:37:21.246888</td>\n",
       "      <td>GOOG</td>\n",
       "      <td>720.50</td>\n",
       "      <td>720.93</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2024-09-11 12:37:21.246888</td>\n",
       "      <td>MSFT</td>\n",
       "      <td>51.95</td>\n",
       "      <td>51.96</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2024-09-11 12:37:21.253888</td>\n",
       "      <td>MSFT</td>\n",
       "      <td>51.97</td>\n",
       "      <td>51.98</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2024-09-11 12:37:21.264888</td>\n",
       "      <td>MSFT</td>\n",
       "      <td>51.99</td>\n",
       "      <td>52.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2024-09-11 12:37:21.271888</td>\n",
       "      <td>GOOG</td>\n",
       "      <td>720.50</td>\n",
       "      <td>720.93</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>2024-09-11 12:37:21.272888</td>\n",
       "      <td>AAPL</td>\n",
       "      <td>97.99</td>\n",
       "      <td>98.01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>2024-09-11 12:37:21.295888</td>\n",
       "      <td>GOOG</td>\n",
       "      <td>720.50</td>\n",
       "      <td>720.88</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>2024-09-11 12:37:21.298888</td>\n",
       "      <td>MSFT</td>\n",
       "      <td>52.01</td>\n",
       "      <td>52.03</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                        time ticker     bid     ask\n",
       "0 2024-09-11 12:37:21.246888   GOOG  720.50  720.93\n",
       "1 2024-09-11 12:37:21.246888   MSFT   51.95   51.96\n",
       "2 2024-09-11 12:37:21.253888   MSFT   51.97   51.98\n",
       "3 2024-09-11 12:37:21.264888   MSFT   51.99   52.00\n",
       "4 2024-09-11 12:37:21.271888   GOOG  720.50  720.93\n",
       "5 2024-09-11 12:37:21.272888   AAPL   97.99   98.01\n",
       "6 2024-09-11 12:37:21.295888   GOOG  720.50  720.88\n",
       "7 2024-09-11 12:37:21.298888   MSFT   52.01   52.03"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "quotes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "tags": [
     "hide-output"
    ]
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>time</th>\n",
       "      <th>ticker</th>\n",
       "      <th>price</th>\n",
       "      <th>quantity</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2024-09-11 12:37:21.275001</td>\n",
       "      <td>MSFT</td>\n",
       "      <td>51.95</td>\n",
       "      <td>75</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2024-09-11 12:37:21.290001</td>\n",
       "      <td>MSFT</td>\n",
       "      <td>51.95</td>\n",
       "      <td>155</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2024-09-11 12:37:21.300001</td>\n",
       "      <td>GOOG</td>\n",
       "      <td>720.77</td>\n",
       "      <td>100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2024-09-11 12:37:21.300001</td>\n",
       "      <td>GOOG</td>\n",
       "      <td>720.92</td>\n",
       "      <td>100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2024-09-11 12:37:21.300001</td>\n",
       "      <td>AAPL</td>\n",
       "      <td>98.00</td>\n",
       "      <td>100</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                        time ticker   price  quantity\n",
       "0 2024-09-11 12:37:21.275001   MSFT   51.95        75\n",
       "1 2024-09-11 12:37:21.290001   MSFT   51.95       155\n",
       "2 2024-09-11 12:37:21.300001   GOOG  720.77       100\n",
       "3 2024-09-11 12:37:21.300001   GOOG  720.92       100\n",
       "4 2024-09-11 12:37:21.300001   AAPL   98.00       100"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "trades"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "tags": [
     "hide-output"
    ]
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ticker</th>\n",
       "      <th>name</th>\n",
       "      <th>exchange</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>MSFT</td>\n",
       "      <td>Microsoft Corporation</td>\n",
       "      <td>NASDAQ</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>GOOG</td>\n",
       "      <td>Alphabet Inc</td>\n",
       "      <td>NASDAQ</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>AAPL</td>\n",
       "      <td>Apple Inc</td>\n",
       "      <td>NASDAQ</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  ticker                   name exchange\n",
       "0   MSFT  Microsoft Corporation   NASDAQ\n",
       "1   GOOG           Alphabet Inc   NASDAQ\n",
       "2   AAPL              Apple Inc   NASDAQ"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "stocks"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Define, infer and ingest feature sets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "import mlrun.feature_store as fstore\n",
    "from mlrun.feature_store.steps import *\n",
    "from mlrun.features import MinMaxValidator"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Build and ingest simple feature set (stocks)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>name</th>\n",
       "      <th>exchange</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ticker</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>MSFT</th>\n",
       "      <td>Microsoft Corporation</td>\n",
       "      <td>NASDAQ</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>GOOG</th>\n",
       "      <td>Alphabet Inc</td>\n",
       "      <td>NASDAQ</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>AAPL</th>\n",
       "      <td>Apple Inc</td>\n",
       "      <td>NASDAQ</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                         name exchange\n",
       "ticker                                \n",
       "MSFT    Microsoft Corporation   NASDAQ\n",
       "GOOG             Alphabet Inc   NASDAQ\n",
       "AAPL                Apple Inc   NASDAQ"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# add feature set without time column (stock ticker metadata)\n",
    "stocks_set = fstore.FeatureSet(\"stocks\", entities=[fstore.Entity(\"ticker\")])\n",
    "fstore.ingest(stocks_set, stocks, infer_options=fstore.InferOptions.default())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Build an advanced feature set - with feature engineering pipeline\n",
    "Define a feature set with custom data processing and time aggregation functions:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "# create a new feature set\n",
    "quotes_set = fstore.FeatureSet(\n",
    "    \"stock-quotes\", entities=[fstore.Entity(\"ticker\")], timestamp_key=\"time\"\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Define a custom pipeline step (python class)**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "class MyMap(MapClass):\n",
    "    def __init__(self, multiplier=1, **kwargs):\n",
    "        super().__init__(**kwargs)\n",
    "        self._multiplier = multiplier\n",
    "\n",
    "    def do(self, event):\n",
    "        event[\"multi\"] = event[\"bid\"] * self._multiplier\n",
    "        return event"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Build and show the transformation pipeline**\n",
    "\n",
    "Use `storey` stream processing classes along with library and custom classes:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       " \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<!-- Generated by graphviz version 2.43.0 (0)\n",
       " -->\n",
       "<!-- Title: mlrun&#45;flow Pages: 1 -->\n",
       "<svg width=\"1008pt\" height=\"98pt\"\n",
       " viewBox=\"0.00 0.00 1008.44 98.00\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
       "<g id=\"graph0\" class=\"graph\" transform=\"scale(1 1) rotate(0) translate(4 94)\">\n",
       "<title>mlrun&#45;flow</title>\n",
       "<polygon fill=\"white\" stroke=\"transparent\" points=\"-4,4 -4,-94 1004.44,-94 1004.44,4 -4,4\"/>\n",
       "<!-- _start -->\n",
       "<g id=\"node1\" class=\"node\">\n",
       "<title>_start</title>\n",
       "<polygon fill=\"lightgrey\" stroke=\"black\" points=\"38.55,-27.05 40.7,-27.15 42.83,-27.3 44.92,-27.49 46.98,-27.74 48.99,-28.03 50.95,-28.36 52.84,-28.75 54.66,-29.18 56.4,-29.65 58.06,-30.16 59.63,-30.71 61.11,-31.31 62.49,-31.94 63.76,-32.61 64.93,-33.31 65.99,-34.04 66.93,-34.8 67.77,-35.59 68.48,-36.41 69.09,-37.25 69.58,-38.11 69.95,-38.99 70.21,-39.89 70.36,-40.8 70.4,-41.72 70.33,-42.65 70.16,-43.59 69.89,-44.53 69.53,-45.47 69.07,-46.41 68.52,-47.35 67.89,-48.28 67.18,-49.2 66.4,-50.11 65.55,-51.01 64.63,-51.89 63.65,-52.75 62.62,-53.59 61.53,-54.41 60.4,-55.2 59.23,-55.96 58.02,-56.69 56.78,-57.39 55.5,-58.06 54.2,-58.69 52.88,-59.29 51.53,-59.84 50.17,-60.35 48.79,-60.82 47.4,-61.25 46,-61.64 44.59,-61.97 43.17,-62.26 41.75,-62.51 40.32,-62.7 38.89,-62.85 37.45,-62.95 36.02,-63 34.58,-63 33.15,-62.95 31.71,-62.85 30.28,-62.7 28.85,-62.51 27.43,-62.26 26.01,-61.97 24.6,-61.64 23.2,-61.25 21.81,-60.82 20.43,-60.35 19.07,-59.84 17.72,-59.29 16.4,-58.69 15.1,-58.06 13.82,-57.39 12.58,-56.69 11.37,-55.96 10.2,-55.2 9.07,-54.41 7.98,-53.59 6.95,-52.75 5.97,-51.89 5.05,-51.01 4.2,-50.11 3.42,-49.2 2.71,-48.28 2.08,-47.35 1.53,-46.41 1.07,-45.47 0.71,-44.53 0.44,-43.59 0.27,-42.65 0.2,-41.72 0.24,-40.8 0.39,-39.89 0.65,-38.99 1.02,-38.11 1.51,-37.25 2.11,-36.41 2.83,-35.59 3.66,-34.8 4.61,-34.04 5.67,-33.31 6.84,-32.61 8.11,-31.94 9.49,-31.31 10.97,-30.71 12.54,-30.16 14.2,-29.65 15.94,-29.18 17.76,-28.75 19.65,-28.36 21.61,-28.03 23.62,-27.74 25.68,-27.49 27.77,-27.3 29.9,-27.15 32.05,-27.05 34.22,-27 36.38,-27 38.55,-27.05\"/>\n",
       "<text text-anchor=\"middle\" x=\"35.3\" y=\"-41.3\" font-family=\"Times,serif\" font-size=\"14.00\">start</text>\n",
       "</g>\n",
       "<!-- MyMap -->\n",
       "<g id=\"node2\" class=\"node\">\n",
       "<title>MyMap</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"152.74\" cy=\"-45\" rx=\"46.29\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"152.74\" y=\"-41.3\" font-family=\"Times,serif\" font-size=\"14.00\">MyMap</text>\n",
       "</g>\n",
       "<!-- _start&#45;&gt;MyMap -->\n",
       "<g id=\"edge1\" class=\"edge\">\n",
       "<title>_start&#45;&gt;MyMap</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M70,-45C78.22,-45 87.23,-45 96.18,-45\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"96.2,-48.5 106.2,-45 96.2,-41.5 96.2,-48.5\"/>\n",
       "</g>\n",
       "<!-- storey.Extend -->\n",
       "<g id=\"node3\" class=\"node\">\n",
       "<title>storey.Extend</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"309.63\" cy=\"-45\" rx=\"74.99\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"309.63\" y=\"-41.3\" font-family=\"Times,serif\" font-size=\"14.00\">storey.Extend</text>\n",
       "</g>\n",
       "<!-- MyMap&#45;&gt;storey.Extend -->\n",
       "<g id=\"edge2\" class=\"edge\">\n",
       "<title>MyMap&#45;&gt;storey.Extend</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M198.96,-45C207.14,-45 215.89,-45 224.76,-45\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"224.8,-48.5 234.8,-45 224.8,-41.5 224.8,-48.5\"/>\n",
       "</g>\n",
       "<!-- filter -->\n",
       "<g id=\"node4\" class=\"node\">\n",
       "<title>filter</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"453.52\" cy=\"-45\" rx=\"33.29\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"453.52\" y=\"-41.3\" font-family=\"Times,serif\" font-size=\"14.00\">filter</text>\n",
       "</g>\n",
       "<!-- storey.Extend&#45;&gt;filter -->\n",
       "<g id=\"edge3\" class=\"edge\">\n",
       "<title>storey.Extend&#45;&gt;filter</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M384.49,-45C393.2,-45 401.85,-45 409.91,-45\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"410.13,-48.5 420.13,-45 410.13,-41.5 410.13,-48.5\"/>\n",
       "</g>\n",
       "<!-- FeaturesetValidator -->\n",
       "<g id=\"node5\" class=\"node\">\n",
       "<title>FeaturesetValidator</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"625.36\" cy=\"-45\" rx=\"102.88\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"625.36\" y=\"-41.3\" font-family=\"Times,serif\" font-size=\"14.00\">FeaturesetValidator</text>\n",
       "</g>\n",
       "<!-- filter&#45;&gt;FeaturesetValidator -->\n",
       "<g id=\"edge4\" class=\"edge\">\n",
       "<title>filter&#45;&gt;FeaturesetValidator</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M486.87,-45C494.57,-45 503.23,-45 512.35,-45\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"512.41,-48.5 522.41,-45 512.41,-41.5 512.41,-48.5\"/>\n",
       "</g>\n",
       "<!-- Aggregates -->\n",
       "<g id=\"node6\" class=\"node\">\n",
       "<title>Aggregates</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"827.75\" cy=\"-45\" rx=\"63.89\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"827.75\" y=\"-41.3\" font-family=\"Times,serif\" font-size=\"14.00\">Aggregates</text>\n",
       "</g>\n",
       "<!-- FeaturesetValidator&#45;&gt;Aggregates -->\n",
       "<g id=\"edge5\" class=\"edge\">\n",
       "<title>FeaturesetValidator&#45;&gt;Aggregates</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M728.32,-45C736.98,-45 745.62,-45 753.99,-45\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"754.04,-48.5 764.04,-45 754.04,-41.5 754.04,-48.5\"/>\n",
       "</g>\n",
       "<!-- parquet/parquet -->\n",
       "<g id=\"node7\" class=\"node\">\n",
       "<title>parquet/parquet</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1000.44,-86.73C1000.44,-88.53 984.08,-90 963.94,-90 943.8,-90 927.44,-88.53 927.44,-86.73 927.44,-86.73 927.44,-57.27 927.44,-57.27 927.44,-55.47 943.8,-54 963.94,-54 984.08,-54 1000.44,-55.47 1000.44,-57.27 1000.44,-57.27 1000.44,-86.73 1000.44,-86.73\"/>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M1000.44,-86.73C1000.44,-84.92 984.08,-83.45 963.94,-83.45 943.8,-83.45 927.44,-84.92 927.44,-86.73\"/>\n",
       "<text text-anchor=\"middle\" x=\"963.94\" y=\"-68.3\" font-family=\"Times,serif\" font-size=\"14.00\">parquet</text>\n",
       "</g>\n",
       "<!-- Aggregates&#45;&gt;parquet/parquet -->\n",
       "<g id=\"edge6\" class=\"edge\">\n",
       "<title>Aggregates&#45;&gt;parquet/parquet</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M880.25,-55.36C892.37,-57.8 905.2,-60.38 917.06,-62.77\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"916.68,-66.26 927.18,-64.8 918.07,-59.4 916.68,-66.26\"/>\n",
       "</g>\n",
       "<!-- nosql/nosql -->\n",
       "<g id=\"node8\" class=\"node\">\n",
       "<title>nosql/nosql</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M991.44,-32.73C991.44,-34.53 979.12,-36 963.94,-36 948.77,-36 936.44,-34.53 936.44,-32.73 936.44,-32.73 936.44,-3.27 936.44,-3.27 936.44,-1.47 948.77,0 963.94,0 979.12,0 991.44,-1.47 991.44,-3.27 991.44,-3.27 991.44,-32.73 991.44,-32.73\"/>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M991.44,-32.73C991.44,-30.92 979.12,-29.45 963.94,-29.45 948.77,-29.45 936.44,-30.92 936.44,-32.73\"/>\n",
       "<text text-anchor=\"middle\" x=\"963.94\" y=\"-14.3\" font-family=\"Times,serif\" font-size=\"14.00\">nosql</text>\n",
       "</g>\n",
       "<!-- Aggregates&#45;&gt;nosql/nosql -->\n",
       "<g id=\"edge7\" class=\"edge\">\n",
       "<title>Aggregates&#45;&gt;nosql/nosql</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M880.25,-34.64C895.59,-31.55 912.09,-28.23 926.28,-25.38\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"927.05,-28.79 936.16,-23.39 925.66,-21.93 927.05,-28.79\"/>\n",
       "</g>\n",
       "</g>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<graphviz.graphs.Digraph at 0x7f572fb4fee0>"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "quotes_set.graph.to(\"MyMap\", multiplier=3).to(\n",
    "    \"storey.Extend\", _fn=\"({'extra': event['bid'] * 77})\"\n",
    ").to(\"storey.Filter\", \"filter\", _fn=\"(event['bid'] > 51.92)\").to(FeaturesetValidator())\n",
    "\n",
    "quotes_set.add_aggregation(\"ask\", [\"sum\", \"max\"], \"1h\", \"10m\", name=\"asks1\")\n",
    "quotes_set.add_aggregation(\"ask\", [\"sum\", \"max\"], \"5h\", \"10m\", name=\"asks5\")\n",
    "quotes_set.add_aggregation(\"bid\", [\"min\", \"max\"], \"1h\", \"10m\", name=\"bids\")\n",
    "\n",
    "# add feature validation policy\n",
    "quotes_set[\"bid\"] = fstore.Feature(validator=MinMaxValidator(min=52, severity=\"info\"))\n",
    "\n",
    "# add default target definitions and plot\n",
    "quotes_set.set_targets()\n",
    "quotes_set.plot(rankdir=\"LR\", with_targets=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Test and show the pipeline results locally (allow to quickly develop and debug)**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "info! bid value is smaller than min, key=['MSFT'] args={'min': 52, 'value': '51.95'}\n",
      "info! bid value is smaller than min, key=['MSFT'] args={'min': 52, 'value': '51.97'}\n",
      "info! bid value is smaller than min, key=['MSFT'] args={'min': 52, 'value': '51.99'}\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>asks1_sum_1h</th>\n",
       "      <th>asks1_max_1h</th>\n",
       "      <th>asks5_sum_5h</th>\n",
       "      <th>asks5_max_5h</th>\n",
       "      <th>bids_min_1h</th>\n",
       "      <th>bids_max_1h</th>\n",
       "      <th>time</th>\n",
       "      <th>bid</th>\n",
       "      <th>ask</th>\n",
       "      <th>multi</th>\n",
       "      <th>extra</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ticker</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>GOOG</th>\n",
       "      <td>720.93</td>\n",
       "      <td>720.93</td>\n",
       "      <td>720.93</td>\n",
       "      <td>720.93</td>\n",
       "      <td>720.50</td>\n",
       "      <td>720.50</td>\n",
       "      <td>2024-09-11 12:37:21.246888</td>\n",
       "      <td>720.50</td>\n",
       "      <td>720.93</td>\n",
       "      <td>2161.50</td>\n",
       "      <td>55478.50</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>MSFT</th>\n",
       "      <td>51.96</td>\n",
       "      <td>51.96</td>\n",
       "      <td>51.96</td>\n",
       "      <td>51.96</td>\n",
       "      <td>51.95</td>\n",
       "      <td>51.95</td>\n",
       "      <td>2024-09-11 12:37:21.246888</td>\n",
       "      <td>51.95</td>\n",
       "      <td>51.96</td>\n",
       "      <td>155.85</td>\n",
       "      <td>4000.15</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>MSFT</th>\n",
       "      <td>103.94</td>\n",
       "      <td>51.98</td>\n",
       "      <td>103.94</td>\n",
       "      <td>51.98</td>\n",
       "      <td>51.95</td>\n",
       "      <td>51.97</td>\n",
       "      <td>2024-09-11 12:37:21.253888</td>\n",
       "      <td>51.97</td>\n",
       "      <td>51.98</td>\n",
       "      <td>155.91</td>\n",
       "      <td>4001.69</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>MSFT</th>\n",
       "      <td>155.94</td>\n",
       "      <td>52.00</td>\n",
       "      <td>155.94</td>\n",
       "      <td>52.00</td>\n",
       "      <td>51.95</td>\n",
       "      <td>51.99</td>\n",
       "      <td>2024-09-11 12:37:21.264888</td>\n",
       "      <td>51.99</td>\n",
       "      <td>52.00</td>\n",
       "      <td>155.97</td>\n",
       "      <td>4003.23</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>GOOG</th>\n",
       "      <td>1441.86</td>\n",
       "      <td>720.93</td>\n",
       "      <td>1441.86</td>\n",
       "      <td>720.93</td>\n",
       "      <td>720.50</td>\n",
       "      <td>720.50</td>\n",
       "      <td>2024-09-11 12:37:21.271888</td>\n",
       "      <td>720.50</td>\n",
       "      <td>720.93</td>\n",
       "      <td>2161.50</td>\n",
       "      <td>55478.50</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>AAPL</th>\n",
       "      <td>98.01</td>\n",
       "      <td>98.01</td>\n",
       "      <td>98.01</td>\n",
       "      <td>98.01</td>\n",
       "      <td>97.99</td>\n",
       "      <td>97.99</td>\n",
       "      <td>2024-09-11 12:37:21.272888</td>\n",
       "      <td>97.99</td>\n",
       "      <td>98.01</td>\n",
       "      <td>293.97</td>\n",
       "      <td>7545.23</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>GOOG</th>\n",
       "      <td>2162.74</td>\n",
       "      <td>720.93</td>\n",
       "      <td>2162.74</td>\n",
       "      <td>720.93</td>\n",
       "      <td>720.50</td>\n",
       "      <td>720.50</td>\n",
       "      <td>2024-09-11 12:37:21.295888</td>\n",
       "      <td>720.50</td>\n",
       "      <td>720.88</td>\n",
       "      <td>2161.50</td>\n",
       "      <td>55478.50</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>MSFT</th>\n",
       "      <td>207.97</td>\n",
       "      <td>52.03</td>\n",
       "      <td>207.97</td>\n",
       "      <td>52.03</td>\n",
       "      <td>51.95</td>\n",
       "      <td>52.01</td>\n",
       "      <td>2024-09-11 12:37:21.298888</td>\n",
       "      <td>52.01</td>\n",
       "      <td>52.03</td>\n",
       "      <td>156.03</td>\n",
       "      <td>4004.77</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        asks1_sum_1h  asks1_max_1h  asks5_sum_5h  asks5_max_5h  bids_min_1h  \\\n",
       "ticker                                                                        \n",
       "GOOG          720.93        720.93        720.93        720.93       720.50   \n",
       "MSFT           51.96         51.96         51.96         51.96        51.95   \n",
       "MSFT          103.94         51.98        103.94         51.98        51.95   \n",
       "MSFT          155.94         52.00        155.94         52.00        51.95   \n",
       "GOOG         1441.86        720.93       1441.86        720.93       720.50   \n",
       "AAPL           98.01         98.01         98.01         98.01        97.99   \n",
       "GOOG         2162.74        720.93       2162.74        720.93       720.50   \n",
       "MSFT          207.97         52.03        207.97         52.03        51.95   \n",
       "\n",
       "        bids_max_1h                       time     bid     ask    multi  \\\n",
       "ticker                                                                    \n",
       "GOOG         720.50 2024-09-11 12:37:21.246888  720.50  720.93  2161.50   \n",
       "MSFT          51.95 2024-09-11 12:37:21.246888   51.95   51.96   155.85   \n",
       "MSFT          51.97 2024-09-11 12:37:21.253888   51.97   51.98   155.91   \n",
       "MSFT          51.99 2024-09-11 12:37:21.264888   51.99   52.00   155.97   \n",
       "GOOG         720.50 2024-09-11 12:37:21.271888  720.50  720.93  2161.50   \n",
       "AAPL          97.99 2024-09-11 12:37:21.272888   97.99   98.01   293.97   \n",
       "GOOG         720.50 2024-09-11 12:37:21.295888  720.50  720.88  2161.50   \n",
       "MSFT          52.01 2024-09-11 12:37:21.298888   52.01   52.03   156.03   \n",
       "\n",
       "           extra  \n",
       "ticker            \n",
       "GOOG    55478.50  \n",
       "MSFT     4000.15  \n",
       "MSFT     4001.69  \n",
       "MSFT     4003.23  \n",
       "GOOG    55478.50  \n",
       "AAPL     7545.23  \n",
       "GOOG    55478.50  \n",
       "MSFT     4004.77  "
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "quotes_set.preview(\n",
    "    quotes,\n",
    "    entity_columns=[\"ticker\"],\n",
    "    options=fstore.InferOptions.default(),\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "tags": [
     "hide-output"
    ]
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "kind: FeatureSet\n",
      "metadata:\n",
      "  name: stock-quotes\n",
      "  project: stocks\n",
      "spec:\n",
      "  entities:\n",
      "  - name: ticker\n",
      "    value_type: str\n",
      "  features:\n",
      "  - name: asks1_sum_1h\n",
      "    value_type: float\n",
      "    aggregate: true\n",
      "  - name: asks1_max_1h\n",
      "    value_type: float\n",
      "    aggregate: true\n",
      "  - name: asks5_sum_5h\n",
      "    value_type: float\n",
      "    aggregate: true\n",
      "  - name: asks5_max_5h\n",
      "    value_type: float\n",
      "    aggregate: true\n",
      "  - name: bids_min_1h\n",
      "    value_type: float\n",
      "    aggregate: true\n",
      "  - name: bids_max_1h\n",
      "    value_type: float\n",
      "    aggregate: true\n",
      "  - name: bid\n",
      "    value_type: float\n",
      "    validator:\n",
      "      kind: minmax\n",
      "      severity: info\n",
      "      min: 52\n",
      "  - name: ask\n",
      "    value_type: float\n",
      "  - name: multi\n",
      "    value_type: float\n",
      "  - name: extra\n",
      "    value_type: float\n",
      "  partition_keys: []\n",
      "  timestamp_key: time\n",
      "  targets:\n",
      "  - name: parquet\n",
      "    kind: parquet\n",
      "    partitioned: true\n",
      "  - name: nosql\n",
      "    kind: nosql\n",
      "    partitioned: false\n",
      "  graph:\n",
      "    steps:\n",
      "      MyMap:\n",
      "        kind: task\n",
      "        class_name: MyMap\n",
      "        class_args:\n",
      "          multiplier: 3\n",
      "        after: []\n",
      "      storey.Extend:\n",
      "        kind: task\n",
      "        class_name: storey.Extend\n",
      "        class_args:\n",
      "          _fn: '({''extra'': event[''bid''] * 77})'\n",
      "        after:\n",
      "        - MyMap\n",
      "      filter:\n",
      "        kind: task\n",
      "        class_name: storey.Filter\n",
      "        class_args:\n",
      "          _fn: (event['bid'] > 51.92)\n",
      "        after:\n",
      "        - storey.Extend\n",
      "      FeaturesetValidator:\n",
      "        kind: task\n",
      "        class_name: mlrun.feature_store.steps.FeaturesetValidator\n",
      "        class_args:\n",
      "          featureset: .\n",
      "        after:\n",
      "        - filter\n",
      "        full_event: true\n",
      "      Aggregates:\n",
      "        kind: task\n",
      "        class_name: storey.AggregateByKey\n",
      "        class_args:\n",
      "          time_field: time\n",
      "          aggregates:\n",
      "          - name: asks1\n",
      "            column: ask\n",
      "            operations:\n",
      "            - sum\n",
      "            - max\n",
      "            windows:\n",
      "            - 1h\n",
      "            period: 10m\n",
      "          - name: asks5\n",
      "            column: ask\n",
      "            operations:\n",
      "            - sum\n",
      "            - max\n",
      "            windows:\n",
      "            - 5h\n",
      "            period: 10m\n",
      "          - name: bids\n",
      "            column: bid\n",
      "            operations:\n",
      "            - min\n",
      "            - max\n",
      "            windows:\n",
      "            - 1h\n",
      "            period: 10m\n",
      "          table: .\n",
      "        after:\n",
      "        - FeaturesetValidator\n",
      "  engine: storey\n",
      "  output_path: v3io:///projects/{{run.project}}/artifacts\n",
      "status:\n",
      "  state: created\n",
      "  stats:\n",
      "    ticker:\n",
      "      count: 8\n",
      "      unique: 3\n",
      "      top: MSFT\n",
      "      freq: 4\n",
      "    asks1_sum_1h:\n",
      "      count: 8.0\n",
      "      mean: 617.9187499999999\n",
      "      min: 51.96\n",
      "      25%: 102.4575\n",
      "      50%: 181.95499999999998\n",
      "      75%: 901.1624999999999\n",
      "      max: 2162.74\n",
      "      std: 784.8779804245735\n",
      "      hist:\n",
      "      - - 4\n",
      "        - 1\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 1\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 1\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 1\n",
      "      - - 51.96\n",
      "        - 157.499\n",
      "        - 263.03799999999995\n",
      "        - 368.57699999999994\n",
      "        - 474.11599999999993\n",
      "        - 579.655\n",
      "        - 685.194\n",
      "        - 790.733\n",
      "        - 896.2719999999999\n",
      "        - 1001.8109999999999\n",
      "        - 1107.35\n",
      "        - 1212.889\n",
      "        - 1318.4279999999999\n",
      "        - 1423.9669999999999\n",
      "        - 1529.5059999999999\n",
      "        - 1635.0449999999998\n",
      "        - 1740.5839999999998\n",
      "        - 1846.1229999999998\n",
      "        - 1951.6619999999998\n",
      "        - 2057.2009999999996\n",
      "        - 2162.74\n",
      "    asks1_max_1h:\n",
      "      count: 8.0\n",
      "      mean: 308.59625\n",
      "      min: 51.96\n",
      "      25%: 51.995\n",
      "      50%: 75.02000000000001\n",
      "      75%: 720.93\n",
      "      max: 720.93\n",
      "      std: 341.7989955655851\n",
      "      hist:\n",
      "      - - 4\n",
      "        - 1\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 3\n",
      "      - - 51.96\n",
      "        - 85.4085\n",
      "        - 118.857\n",
      "        - 152.3055\n",
      "        - 185.754\n",
      "        - 219.2025\n",
      "        - 252.65099999999998\n",
      "        - 286.0995\n",
      "        - 319.54799999999994\n",
      "        - 352.9964999999999\n",
      "        - 386.44499999999994\n",
      "        - 419.89349999999996\n",
      "        - 453.3419999999999\n",
      "        - 486.7904999999999\n",
      "        - 520.2389999999999\n",
      "        - 553.6875\n",
      "        - 587.136\n",
      "        - 620.5844999999999\n",
      "        - 654.0329999999999\n",
      "        - 687.4815\n",
      "        - 720.93\n",
      "    asks5_sum_5h:\n",
      "      count: 8.0\n",
      "      mean: 617.9187499999999\n",
      "      min: 51.96\n",
      "      25%: 102.4575\n",
      "      50%: 181.95499999999998\n",
      "      75%: 901.1624999999999\n",
      "      max: 2162.74\n",
      "      std: 784.8779804245735\n",
      "      hist:\n",
      "      - - 4\n",
      "        - 1\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 1\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 1\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 1\n",
      "      - - 51.96\n",
      "        - 157.499\n",
      "        - 263.03799999999995\n",
      "        - 368.57699999999994\n",
      "        - 474.11599999999993\n",
      "        - 579.655\n",
      "        - 685.194\n",
      "        - 790.733\n",
      "        - 896.2719999999999\n",
      "        - 1001.8109999999999\n",
      "        - 1107.35\n",
      "        - 1212.889\n",
      "        - 1318.4279999999999\n",
      "        - 1423.9669999999999\n",
      "        - 1529.5059999999999\n",
      "        - 1635.0449999999998\n",
      "        - 1740.5839999999998\n",
      "        - 1846.1229999999998\n",
      "        - 1951.6619999999998\n",
      "        - 2057.2009999999996\n",
      "        - 2162.74\n",
      "    asks5_max_5h:\n",
      "      count: 8.0\n",
      "      mean: 308.59625\n",
      "      min: 51.96\n",
      "      25%: 51.995\n",
      "      50%: 75.02000000000001\n",
      "      75%: 720.93\n",
      "      max: 720.93\n",
      "      std: 341.7989955655851\n",
      "      hist:\n",
      "      - - 4\n",
      "        - 1\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 3\n",
      "      - - 51.96\n",
      "        - 85.4085\n",
      "        - 118.857\n",
      "        - 152.3055\n",
      "        - 185.754\n",
      "        - 219.2025\n",
      "        - 252.65099999999998\n",
      "        - 286.0995\n",
      "        - 319.54799999999994\n",
      "        - 352.9964999999999\n",
      "        - 386.44499999999994\n",
      "        - 419.89349999999996\n",
      "        - 453.3419999999999\n",
      "        - 486.7904999999999\n",
      "        - 520.2389999999999\n",
      "        - 553.6875\n",
      "        - 587.136\n",
      "        - 620.5844999999999\n",
      "        - 654.0329999999999\n",
      "        - 687.4815\n",
      "        - 720.93\n",
      "    bids_min_1h:\n",
      "      count: 8.0\n",
      "      mean: 308.41125\n",
      "      min: 51.95\n",
      "      25%: 51.95\n",
      "      50%: 74.97\n",
      "      75%: 720.5\n",
      "      max: 720.5\n",
      "      std: 341.59667259325835\n",
      "      hist:\n",
      "      - - 4\n",
      "        - 1\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 3\n",
      "      - - 51.95\n",
      "        - 85.3775\n",
      "        - 118.80499999999999\n",
      "        - 152.2325\n",
      "        - 185.65999999999997\n",
      "        - 219.08749999999998\n",
      "        - 252.515\n",
      "        - 285.94249999999994\n",
      "        - 319.36999999999995\n",
      "        - 352.79749999999996\n",
      "        - 386.22499999999997\n",
      "        - 419.6524999999999\n",
      "        - 453.0799999999999\n",
      "        - 486.50749999999994\n",
      "        - 519.935\n",
      "        - 553.3625\n",
      "        - 586.79\n",
      "        - 620.2175\n",
      "        - 653.645\n",
      "        - 687.0725\n",
      "        - 720.5\n",
      "    bids_max_1h:\n",
      "      count: 8.0\n",
      "      mean: 308.42625\n",
      "      min: 51.95\n",
      "      25%: 51.985\n",
      "      50%: 75.0\n",
      "      75%: 720.5\n",
      "      max: 720.5\n",
      "      std: 341.58380276661245\n",
      "      hist:\n",
      "      - - 4\n",
      "        - 1\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 3\n",
      "      - - 51.95\n",
      "        - 85.3775\n",
      "        - 118.80499999999999\n",
      "        - 152.2325\n",
      "        - 185.65999999999997\n",
      "        - 219.08749999999998\n",
      "        - 252.515\n",
      "        - 285.94249999999994\n",
      "        - 319.36999999999995\n",
      "        - 352.79749999999996\n",
      "        - 386.22499999999997\n",
      "        - 419.6524999999999\n",
      "        - 453.0799999999999\n",
      "        - 486.50749999999994\n",
      "        - 519.935\n",
      "        - 553.3625\n",
      "        - 586.79\n",
      "        - 620.2175\n",
      "        - 653.645\n",
      "        - 687.0725\n",
      "        - 720.5\n",
      "    time:\n",
      "      count: 8\n",
      "      mean: '2024-09-11 12:37:21.269012992'\n",
      "      min: '2024-09-11 12:37:21.246888'\n",
      "      25%: '2024-09-11 12:37:21.252137984'\n",
      "      50%: '2024-09-11 12:37:21.268387840'\n",
      "      75%: '2024-09-11 12:37:21.278638080'\n",
      "      max: '2024-09-11 12:37:21.298888'\n",
      "    bid:\n",
      "      count: 8.0\n",
      "      mean: 308.42625\n",
      "      min: 51.95\n",
      "      25%: 51.985\n",
      "      50%: 75.0\n",
      "      75%: 720.5\n",
      "      max: 720.5\n",
      "      std: 341.58380276661245\n",
      "      hist:\n",
      "      - - 4\n",
      "        - 1\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 3\n",
      "      - - 51.95\n",
      "        - 85.3775\n",
      "        - 118.80499999999999\n",
      "        - 152.2325\n",
      "        - 185.65999999999997\n",
      "        - 219.08749999999998\n",
      "        - 252.515\n",
      "        - 285.94249999999994\n",
      "        - 319.36999999999995\n",
      "        - 352.79749999999996\n",
      "        - 386.22499999999997\n",
      "        - 419.6524999999999\n",
      "        - 453.0799999999999\n",
      "        - 486.50749999999994\n",
      "        - 519.935\n",
      "        - 553.3625\n",
      "        - 586.79\n",
      "        - 620.2175\n",
      "        - 653.645\n",
      "        - 687.0725\n",
      "        - 720.5\n",
      "    ask:\n",
      "      count: 8.0\n",
      "      mean: 308.59\n",
      "      min: 51.96\n",
      "      25%: 51.995\n",
      "      50%: 75.02000000000001\n",
      "      75%: 720.8924999999999\n",
      "      max: 720.93\n",
      "      std: 341.79037903369954\n",
      "      hist:\n",
      "      - - 4\n",
      "        - 1\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 3\n",
      "      - - 51.96\n",
      "        - 85.4085\n",
      "        - 118.857\n",
      "        - 152.3055\n",
      "        - 185.754\n",
      "        - 219.2025\n",
      "        - 252.65099999999998\n",
      "        - 286.0995\n",
      "        - 319.54799999999994\n",
      "        - 352.9964999999999\n",
      "        - 386.44499999999994\n",
      "        - 419.89349999999996\n",
      "        - 453.3419999999999\n",
      "        - 486.7904999999999\n",
      "        - 520.2389999999999\n",
      "        - 553.6875\n",
      "        - 587.136\n",
      "        - 620.5844999999999\n",
      "        - 654.0329999999999\n",
      "        - 687.4815\n",
      "        - 720.93\n",
      "    multi:\n",
      "      count: 8.0\n",
      "      mean: 925.27875\n",
      "      min: 155.85000000000002\n",
      "      25%: 155.95499999999998\n",
      "      50%: 225.0\n",
      "      75%: 2161.5\n",
      "      max: 2161.5\n",
      "      std: 1024.7514082998375\n",
      "      hist:\n",
      "      - - 4\n",
      "        - 1\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 3\n",
      "      - - 155.85000000000002\n",
      "        - 256.13250000000005\n",
      "        - 356.415\n",
      "        - 456.6975\n",
      "        - 556.98\n",
      "        - 657.2625\n",
      "        - 757.545\n",
      "        - 857.8275\n",
      "        - 958.11\n",
      "        - 1058.3925\n",
      "        - 1158.6750000000002\n",
      "        - 1258.9575\n",
      "        - 1359.2399999999998\n",
      "        - 1459.5225\n",
      "        - 1559.8049999999998\n",
      "        - 1660.0875\n",
      "        - 1760.37\n",
      "        - 1860.6525000000001\n",
      "        - 1960.935\n",
      "        - 2061.2175\n",
      "        - 2161.5\n",
      "    extra:\n",
      "      count: 8.0\n",
      "      mean: 23748.82125\n",
      "      min: 4000.15\n",
      "      25%: 4002.8450000000003\n",
      "      50%: 5775.0\n",
      "      75%: 55478.5\n",
      "      max: 55478.5\n",
      "      std: 26301.95281302916\n",
      "      hist:\n",
      "      - - 4\n",
      "        - 1\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 0\n",
      "        - 3\n",
      "      - - 4000.15\n",
      "        - 6574.0675\n",
      "        - 9147.985\n",
      "        - 11721.9025\n",
      "        - 14295.82\n",
      "        - 16869.7375\n",
      "        - 19443.655000000002\n",
      "        - 22017.572500000002\n",
      "        - 24591.49\n",
      "        - 27165.4075\n",
      "        - 29739.325\n",
      "        - 32313.2425\n",
      "        - 34887.16\n",
      "        - 37461.0775\n",
      "        - 40034.995\n",
      "        - 42608.9125\n",
      "        - 45182.83\n",
      "        - 47756.747500000005\n",
      "        - 50330.665\n",
      "        - 52904.582500000004\n",
      "        - 55478.5\n",
      "  preview:\n",
      "  - - ticker\n",
      "    - asks1_sum_1h\n",
      "    - asks1_max_1h\n",
      "    - asks5_sum_5h\n",
      "    - asks5_max_5h\n",
      "    - bids_min_1h\n",
      "    - bids_max_1h\n",
      "    - time\n",
      "    - bid\n",
      "    - ask\n",
      "    - multi\n",
      "    - extra\n",
      "  - - GOOG\n",
      "    - 720.93\n",
      "    - 720.93\n",
      "    - 720.93\n",
      "    - 720.93\n",
      "    - 720.5\n",
      "    - 720.5\n",
      "    - 2024-09-11T12:37:21.246888\n",
      "    - 720.5\n",
      "    - 720.93\n",
      "    - 2161.5\n",
      "    - 55478.5\n",
      "  - - MSFT\n",
      "    - 51.96\n",
      "    - 51.96\n",
      "    - 51.96\n",
      "    - 51.96\n",
      "    - 51.95\n",
      "    - 51.95\n",
      "    - 2024-09-11T12:37:21.246888\n",
      "    - 51.95\n",
      "    - 51.96\n",
      "    - 155.85000000000002\n",
      "    - 4000.15\n",
      "  - - MSFT\n",
      "    - 103.94\n",
      "    - 51.98\n",
      "    - 103.94\n",
      "    - 51.98\n",
      "    - 51.95\n",
      "    - 51.97\n",
      "    - 2024-09-11T12:37:21.253888\n",
      "    - 51.97\n",
      "    - 51.98\n",
      "    - 155.91\n",
      "    - 4001.69\n",
      "  - - MSFT\n",
      "    - 155.94\n",
      "    - 52.0\n",
      "    - 155.94\n",
      "    - 52.0\n",
      "    - 51.95\n",
      "    - 51.99\n",
      "    - 2024-09-11T12:37:21.264888\n",
      "    - 51.99\n",
      "    - 52.0\n",
      "    - 155.97\n",
      "    - 4003.23\n",
      "  - - GOOG\n",
      "    - 1441.86\n",
      "    - 720.93\n",
      "    - 1441.86\n",
      "    - 720.93\n",
      "    - 720.5\n",
      "    - 720.5\n",
      "    - 2024-09-11T12:37:21.271888\n",
      "    - 720.5\n",
      "    - 720.93\n",
      "    - 2161.5\n",
      "    - 55478.5\n",
      "  - - AAPL\n",
      "    - 98.01\n",
      "    - 98.01\n",
      "    - 98.01\n",
      "    - 98.01\n",
      "    - 97.99\n",
      "    - 97.99\n",
      "    - 2024-09-11T12:37:21.272888\n",
      "    - 97.99\n",
      "    - 98.01\n",
      "    - 293.96999999999997\n",
      "    - 7545.23\n",
      "  - - GOOG\n",
      "    - 2162.74\n",
      "    - 720.93\n",
      "    - 2162.74\n",
      "    - 720.93\n",
      "    - 720.5\n",
      "    - 720.5\n",
      "    - 2024-09-11T12:37:21.295888\n",
      "    - 720.5\n",
      "    - 720.88\n",
      "    - 2161.5\n",
      "    - 55478.5\n",
      "  - - MSFT\n",
      "    - 207.97\n",
      "    - 52.03\n",
      "    - 207.97\n",
      "    - 52.03\n",
      "    - 51.95\n",
      "    - 52.01\n",
      "    - 2024-09-11T12:37:21.298888\n",
      "    - 52.01\n",
      "    - 52.03\n",
      "    - 156.03\n",
      "    - 4004.77\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# print the feature set object\n",
    "print(quotes_set.to_yaml())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Ingest data into offline and online stores\n",
    "This writes to both targets (Parquet and NoSQL)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "info! bid value is smaller than min, key=['MSFT'] args={'min': 52, 'value': '51.95'}\n",
      "info! bid value is smaller than min, key=['MSFT'] args={'min': 52, 'value': '51.97'}\n",
      "info! bid value is smaller than min, key=['MSFT'] args={'min': 52, 'value': '51.99'}\n",
      "info! bid value is smaller than min, key=['MSFT'] args={'min': 52, 'value': '51.95'}\n",
      "info! bid value is smaller than min, key=['MSFT'] args={'min': 52, 'value': '51.97'}\n",
      "info! bid value is smaller than min, key=['MSFT'] args={'min': 52, 'value': '51.99'}\n"
     ]
    }
   ],
   "source": [
    "# save ingest data and print the FeatureSet spec\n",
    "df = quotes_set.ingest(quotes)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Get an offline feature vector for training\n",
    "Example of combining features from 3 sources with time travel join of 3 tables with **time travel**.\n",
    "\n",
    "Specify a set of features and request the feature vector offline result as a dataframe:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "features = [\n",
    "    \"stock-quotes.multi\",\n",
    "    \"stock-quotes.asks5_sum_5h as total_ask\",\n",
    "    \"stock-quotes.bids_min_1h\",\n",
    "    \"stock-quotes.bids_max_1h\",\n",
    "    \"stocks.*\",\n",
    "]\n",
    "\n",
    "vector = fstore.FeatureVector(\n",
    "    \"stocks-vec\", features, description=\"stocks demo feature vector\"\n",
    ")\n",
    "vector.save()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "> 2024-09-11 12:37:23,529 [info] Merger detected timestamp resolution incompatibility between feature set stock-quotes and others: datetime64[ns] and datetime64[us]. Converting feature set timestamp column 'time' to type datetime64[ns].\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>price</th>\n",
       "      <th>quantity</th>\n",
       "      <th>multi</th>\n",
       "      <th>total_ask</th>\n",
       "      <th>bids_min_1h</th>\n",
       "      <th>bids_max_1h</th>\n",
       "      <th>name</th>\n",
       "      <th>exchange</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>51.95</td>\n",
       "      <td>75</td>\n",
       "      <td>155.97</td>\n",
       "      <td>155.94</td>\n",
       "      <td>51.95</td>\n",
       "      <td>51.99</td>\n",
       "      <td>Microsoft Corporation</td>\n",
       "      <td>NASDAQ</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>51.95</td>\n",
       "      <td>155</td>\n",
       "      <td>155.97</td>\n",
       "      <td>155.94</td>\n",
       "      <td>51.95</td>\n",
       "      <td>51.99</td>\n",
       "      <td>Microsoft Corporation</td>\n",
       "      <td>NASDAQ</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>720.77</td>\n",
       "      <td>100</td>\n",
       "      <td>2161.50</td>\n",
       "      <td>2162.74</td>\n",
       "      <td>720.50</td>\n",
       "      <td>720.50</td>\n",
       "      <td>Alphabet Inc</td>\n",
       "      <td>NASDAQ</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>720.92</td>\n",
       "      <td>100</td>\n",
       "      <td>2161.50</td>\n",
       "      <td>2162.74</td>\n",
       "      <td>720.50</td>\n",
       "      <td>720.50</td>\n",
       "      <td>Alphabet Inc</td>\n",
       "      <td>NASDAQ</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>98.00</td>\n",
       "      <td>100</td>\n",
       "      <td>293.97</td>\n",
       "      <td>98.01</td>\n",
       "      <td>97.99</td>\n",
       "      <td>97.99</td>\n",
       "      <td>Apple Inc</td>\n",
       "      <td>NASDAQ</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    price  quantity    multi  total_ask  bids_min_1h  bids_max_1h  \\\n",
       "0   51.95        75   155.97     155.94        51.95        51.99   \n",
       "1   51.95       155   155.97     155.94        51.95        51.99   \n",
       "2  720.77       100  2161.50    2162.74       720.50       720.50   \n",
       "3  720.92       100  2161.50    2162.74       720.50       720.50   \n",
       "4   98.00       100   293.97      98.01        97.99        97.99   \n",
       "\n",
       "                    name exchange  \n",
       "0  Microsoft Corporation   NASDAQ  \n",
       "1  Microsoft Corporation   NASDAQ  \n",
       "2           Alphabet Inc   NASDAQ  \n",
       "3           Alphabet Inc   NASDAQ  \n",
       "4              Apple Inc   NASDAQ  "
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "resp = vector.get_offline_features(entity_rows=trades, entity_timestamp_column=\"time\")\n",
    "resp.to_dataframe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Initialize an online feature service and use it for real-time inference"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "service = vector.get_online_feature_service(\"\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Request feature vector statistics, can be used for imputing or validation**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>count</th>\n",
       "      <th>mean</th>\n",
       "      <th>min</th>\n",
       "      <th>25%</th>\n",
       "      <th>50%</th>\n",
       "      <th>75%</th>\n",
       "      <th>max</th>\n",
       "      <th>std</th>\n",
       "      <th>hist</th>\n",
       "      <th>unique</th>\n",
       "      <th>top</th>\n",
       "      <th>freq</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>multi</th>\n",
       "      <td>8.0</td>\n",
       "      <td>925.27875</td>\n",
       "      <td>155.85</td>\n",
       "      <td>155.9550</td>\n",
       "      <td>225.000</td>\n",
       "      <td>2161.5000</td>\n",
       "      <td>2161.50</td>\n",
       "      <td>1024.751408</td>\n",
       "      <td>[[4, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>bids_min_1h</th>\n",
       "      <td>8.0</td>\n",
       "      <td>308.41125</td>\n",
       "      <td>51.95</td>\n",
       "      <td>51.9500</td>\n",
       "      <td>74.970</td>\n",
       "      <td>720.5000</td>\n",
       "      <td>720.50</td>\n",
       "      <td>341.596673</td>\n",
       "      <td>[[4, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>bids_max_1h</th>\n",
       "      <td>8.0</td>\n",
       "      <td>308.42625</td>\n",
       "      <td>51.95</td>\n",
       "      <td>51.9850</td>\n",
       "      <td>75.000</td>\n",
       "      <td>720.5000</td>\n",
       "      <td>720.50</td>\n",
       "      <td>341.583803</td>\n",
       "      <td>[[4, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>name</th>\n",
       "      <td>3.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3.0</td>\n",
       "      <td>Microsoft Corporation</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>exchange</th>\n",
       "      <td>3.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NASDAQ</td>\n",
       "      <td>3.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>total_ask</th>\n",
       "      <td>8.0</td>\n",
       "      <td>617.91875</td>\n",
       "      <td>51.96</td>\n",
       "      <td>102.4575</td>\n",
       "      <td>181.955</td>\n",
       "      <td>901.1625</td>\n",
       "      <td>2162.74</td>\n",
       "      <td>784.877980</td>\n",
       "      <td>[[4, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 1, 0,...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             count       mean     min       25%      50%        75%      max  \\\n",
       "multi          8.0  925.27875  155.85  155.9550  225.000  2161.5000  2161.50   \n",
       "bids_min_1h    8.0  308.41125   51.95   51.9500   74.970   720.5000   720.50   \n",
       "bids_max_1h    8.0  308.42625   51.95   51.9850   75.000   720.5000   720.50   \n",
       "name           3.0        NaN     NaN       NaN      NaN        NaN      NaN   \n",
       "exchange       3.0        NaN     NaN       NaN      NaN        NaN      NaN   \n",
       "total_ask      8.0  617.91875   51.96  102.4575  181.955   901.1625  2162.74   \n",
       "\n",
       "                     std                                               hist  \\\n",
       "multi        1024.751408  [[4, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,...   \n",
       "bids_min_1h   341.596673  [[4, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,...   \n",
       "bids_max_1h   341.583803  [[4, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,...   \n",
       "name                 NaN                                                NaN   \n",
       "exchange             NaN                                                NaN   \n",
       "total_ask     784.877980  [[4, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 1, 0,...   \n",
       "\n",
       "             unique                    top  freq  \n",
       "multi           NaN                    NaN   NaN  \n",
       "bids_min_1h     NaN                    NaN   NaN  \n",
       "bids_max_1h     NaN                    NaN   NaN  \n",
       "name            3.0  Microsoft Corporation   1.0  \n",
       "exchange        1.0                 NASDAQ   3.0  \n",
       "total_ask       NaN                    NaN   NaN  "
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "service.vector.get_stats_table()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Real-time feature vector request**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'bids_min_1h': 720.5,\n",
       "  'bids_max_1h': 720.5,\n",
       "  'total_ask': 2162.74,\n",
       "  'multi': 2161.5,\n",
       "  'name': 'Alphabet Inc',\n",
       "  'exchange': 'NASDAQ'},\n",
       " {'bids_min_1h': 51.95,\n",
       "  'bids_max_1h': 52.01,\n",
       "  'total_ask': 207.97,\n",
       "  'multi': 156.03,\n",
       "  'name': 'Microsoft Corporation',\n",
       "  'exchange': 'NASDAQ'}]"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "service.get([{\"ticker\": \"GOOG\"}, {\"ticker\": \"MSFT\"}])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'bids_min_1h': 97.99,\n",
       "  'bids_max_1h': 97.99,\n",
       "  'total_ask': 98.01,\n",
       "  'multi': 293.97,\n",
       "  'name': 'Apple Inc',\n",
       "  'exchange': 'NASDAQ'}]"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "service.get([{\"ticker\": \"AAPL\"}])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "service.close()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "mlrun-base",
   "language": "python",
   "name": "conda-env-mlrun-base-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.18"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
